1. Field of the Invention
This invention relates to a neural network, particularly to a VLSI implementation of neural networks. It shortens the convergence time and eliminates the influence of the initial value by using the GA features, thereby achieving the conservation of hardware resources.
2. Description of the Related Art
Because of its independence on specific model of signal processing, artificial neural networks (ANN) is now widely used in the fields of image processing and pattern recognition. Refer to FIG. 1. A multilayer neural network typically comprising an input layer, an output layer, and a hidden layer. The input layer having a row of cells 1, . . . , 2 or neuron-like units (not shown) serves as an interconnect function, externally connecting a plurality of inputs x1, . . . , xn to the network, wherein each input is coupled to a respective input cell. The hidden layer having a plurality of neuron cells 3, . . . , 4, which is the most important part in the network and has multiple inputs and one output, is responsible for performing the specific functions of the network, wherein the output of each input cell in the input layer is coupled to each of a plurality of neuron cells in the hidden layer. For example, the specific functions can be that each neuron cell multiplies the inputs x1, . . . , xn by a given weight w1, w2, . . . , w3, w4 and adds the products together, as shown in 3 and 4 in the hidden layer. The output layer having at least one cell 5 or neuron-like unit (not shown) serves as the output collection of a plurality of neuron cells 3, . . . , 4 in the hidden layer. For such a network, although now the majorities of neural systems are realized by software simulations, only hardware implementations can fully utilize its advantages of parallel processing and error tolerance. Among the existing 3 methods of VLSI implementation for neural systems, analog circuit has the merits of small chip area, fast speed and low power consumption, but is susceptible to process parameters and interface noises. Digital circuit has the merits of high precision and robustness, but it is area and power consuming. The digit-analog hybrid methodxe2x80x94pulse stream technique combines both the merits of analog and digital methods. It uses digital signal to control analog multiplication, so the area of synapse analog multiplier is small and the neuron digital state signal is immune to noise. Among the various kinds of pulse modulation techniques, pulse width modulation (PWM) is the easiest for circuit implementation, so the PWM VLSI neural network is widely studied.
Although the first proposed structure of 3-transistor synapse multiplier is simple, it has low precision and small linearity range because of the switching noise and working states of the transistors. Furthermore, it uses external non-linear voltages for activation function, which is not suitable for VLSI implementation because of the limited interface. A low gain 2-transistor amplifier is adopted for activation function, but its transfer characteristic is quite different from the ideal sigmoid function, so it is not suitable for ANNs which use Back Propagation (BP) learning algorithm. Although the transfer characteristic of asynchronous neuron proposed is very similar to ideal sigmoid function, it needs complex BiCMOS process. Another novel neuron circuit can easily realize both sigmoid function and its derivative, but its current input is not suitable for voltage-mode sigmoid circuit.
Therefore, an object of the invention is to provide a new building blocks for PWM VLSI neural network, including a simple precise synapse multiplier with large linearity range and a voltages-pulse converter with high precision.
Another object of the invention is to provide a CMOS voltage-mode sigmoid circuit with adjustable gain, whose transfer function is similar to ideal sigmoid function.
To realize the above and other objects, the invention provides a voltage-mode pulse width modulation (PWM) VLSI implementation of neural networks, including a simple precise synapse multiplier with large linearity range and a voltages-pulse converter with high precision. Further, A CMOS voltage-mode sigmoid circuit with adjustable gain is designed, whose transfer function is similar to ideal sigmoid function, thereby constructing a 2-2-1 feedforward PWM ANN.